1 Objetivo

O objetivo desse notebook é efetuar todo o processo de modelagem da base de dados adult, disponibilizada para o desafio do curso de introdução ao Machine Learning da Curso-R, utilizando o framework tidymodels. Ou seja, explorar, tratar, preparar, tunnar e escolher o modelo que melhor se ajusta aos dados disponibilizados.

2 Leitura da base

2.1 Informações preliminares

adult <- read_rds("adult.rds")

# head(adult) 

# glimpse(adult)
skim(adult)
-- Data Summary ------------------------
                           Values
Name                       adult 
Number of rows             32561 
Number of columns          16    
_______________________          
Column type frequency:           
  character                9     
  numeric                  7     
________________________         
Group variables            None  

-- Variable type: character ----------------------------------------------------------------------------------------
# A tibble: 9 x 8
  skim_variable  n_missing complete_rate   min   max empty n_unique whitespace
* <chr>              <int>         <dbl> <int> <int> <int>    <int>      <int>
1 workclass           1836         0.944     7    16     0        8          0
2 education              0         1         3    12     0       16          0
3 marital_status         0         1         7    21     0        7          0
4 occupation          1843         0.943     5    17     0       14          0
5 relationship           0         1         4    14     0        6          0
6 race                   0         1         5    18     0        5          0
7 sex                    0         1         4     6     0        2          0
8 native_country       583         0.982     4    26     0       41          0
9 resposta               0         1         4     5     0        2          0

-- Variable type: numeric ------------------------------------------------------------------------------------------
# A tibble: 7 x 11
  skim_variable  n_missing complete_rate     mean        sd    p0    p25    p50    p75    p100 hist 
* <chr>              <int>         <dbl>    <dbl>     <dbl> <dbl>  <dbl>  <dbl>  <dbl>   <dbl> <chr>
1 age                    0             1     38.6     13.6     17     28     37     48      90 ▇▇▅▂▁
2 fnlwgt                 0             1 189778.  105550.   12285 117827 178356 237051 1484705 ▇▁▁▁▁
3 education_num          0             1     10.1      2.57     1      9     10     12      16 ▁▁▇▃▁
4 capital_gain           0             1   1078.    7385.       0      0      0      0   99999 ▇▁▁▁▁
5 capital_loss           0             1     87.3    403.       0      0      0      0    4356 ▇▁▁▁▁
6 hours_per_week         0             1     40.4     12.3      1     40     40     45      99 ▁▇▃▁▁
7 id                     0             1  16281     9400.       1   8141  16281  24421   32561 ▇▇▇▇▇


As variáveis parecem estar com formatos corretos. Ponto de atenção para as variáveis wokclass, occupation e native_country, que apresentam valores missing.

3 AED


Agora vamos analisar o comportamento das variáveis para definirmos como tratar os nossos dados para o modelo.

3.1 Parte 1



# DataExplorer::create_report(adult)

devtools::source_url("https://raw.githubusercontent.com/ricardomattos05/functions/master/function_AED_bivariada.R")
# 
# 
adult2 <- adult %>%
            select(-id) %>% 
            mutate(resposta = if_else(resposta == ">50K", 1, 0))
# 
# 

# names(adult2)
for (i in 1:(length(adult2)-1) ) {
  
  df <- adult2[,c(i,15)]
  cat("### ",names(df[,1]),"\n") 
  print(AED_biv(df,glue("resposta"),"Pre"))
  cat('\n\n')
}

3.1.1 age

NULL

3.1.2 workclass

NULL

3.1.3 fnlwgt

NULL

3.1.4 education

NULL

3.1.5 education_num

NULL

3.1.6 marital_status

NULL

3.1.7 occupation

NULL

3.1.8 relationship

NULL

3.1.9 race

NULL

3.1.10 sex

NULL

3.1.11 capital_gain

NULL

3.1.12 capital_loss

NULL

3.1.13 hours_per_week

NULL

3.1.14 native_country

NULL

Observações:

  • education : é possível visualizar que quanto maior o grau de escolaridade, maior a proporção de pessoas com salarios acima de 50k. E que as categorias abaixo de HS-grad, 1th-4th até 12thalém de serem pouco representativas, possuem baixa proporção, vamos então criar uma categoria uma nova consolidando elas HS-not-grad.

  • marital_status : aqui iremos agrupar os campos Married-AF-spouse e Married-civ-spouse, criando a categoria Married, baseado na similaridade entre elas com relação a variável resposta e considerando a descrição delas.

  • native_country : É um campo com pouca variabilidade, onde 90% dos dados estão atribuídos como “Estados Unidos”. Sendo assim, poderia considerar apenas Estados Unidos e agrupar o restante como outros, mas vamos manter o máximo de informação e reduzir as categorias para 3, agrupando todos os países que obtiveram proporção maior que a média, manter o valor mais representativo e uma categoria com os países abaixo da média.

  • relationship : campo contém os campos husband e wife, aparentemente poderiamos agrupa-los, vamos analisar mais afundo.

  • capital_loss e capital_gain : Aparentemente tanto quem ganha quanto quem perde algum valor apresentam maiores probabilidades de ter salario >50k. Vamos então avaliar a correlação entre elas.

  • workclass : Categorias com baixa representatividade como Never-workede Without-pay não possuem classificação com a resposta de interesse “>50k”, vamos dar um zoom nessa variável e analisar os NA’s que identificamos também.

3.2 Parte 2

3.2.1 occupation


ggplot(adult, aes(x = occupation, fill = resposta)) + 
  geom_bar(position="fill") + 
  theme(axis.text.x = element_text(angle = 90)) + 
  ggtitle("occupation")


É possível ver que não faria sentido atribuir os NAs de forma modal, uma vez que nosso objetivo é obter o maior poder preditivo possível, logo, não queremos perder informação. Sendo assim, não vamos diluir os NAs na categoria com maior representatividade Prof-specialty, vamos atribuir à uma categoria com proporções similares e que possui uma boa representatividade, Farming-fishing.

3.2.2 relationship


ggplot(adult, aes(x = relationship)) +
  geom_bar() +
  theme(axis.text.x = element_text(angle = 90)) + 
  ggtitle("relationship")


ggplot(adult, aes(x = relationship, fill = resposta)) + 
  geom_bar(position="fill") + 
  theme(axis.text.x = element_text(angle = 90)) + 
  ggtitle("relationship")


ggplot(adult, aes(x = relationship, fill = sex)) + 
  geom_bar(position="fill") + 
  theme(axis.text.x = element_text(angle = 90)) + 
  ggtitle("relationship")

Vamos então balancear o gênero agrupando as categorias Wife e Husband, criando a categoria Married.

3.2.3 Capital Gain and Loss


ggplot(adult, aes(x= capital_gain, y= capital_loss)) +
  geom_point()

sum(adult$capital_loss > 0 & adult$capital_gain > 0)
[1] 0

Sendo assim, podemos soma-las e criar a variável capital_total sem medo de perder informação.

3.2.4 Worclass


ggplot(adult, aes(x = workclass)) +
  geom_bar() +
  theme(axis.text.x = element_text(angle = 90)) + 
  ggtitle("Workclass")


ggplot(adult, aes(x = workclass, fill = resposta)) + 
  geom_bar(position="fill") + 
  theme(axis.text.x = element_text(angle = 90)) + 
  ggtitle("Workclass")

Pelo visto a catgoria NA possui relação com a variável resposta distinta de todas as outras categorias, vamos então gerar uma nova categoria not-identify para atribuir os valores NA.

3.2.5 native_country


med <- (adult %>% 
          select(resposta) %>%
          filter(resposta == ">50K") %>% 
          count() %>% 
          as.numeric())/nrow(adult)
         

tb_country<- adult %>% 
                select(native_country, resposta) %>% 
                group_by(native_country) %>% 
                count(resposta) %>% 
                mutate(prop = prop.table(n)) %>% 
                filter(resposta == ">50K") %>% 
                mutate( class = case_when( native_country == "United-States" ~ "United-States",
                                           prop > med ~ ">mean",
                                           prop <= med ~ "<=mean" )  )

tb_country %>% 
  select(native_country,class) %>% 
  group_by(class) %>% 
  count()
NA
NA
NA

Ficamos então com 21 países com proporções abaixo da méda, 18 acima e “United-States” como as 3 categorias restantes.


A distribuição ficou com 5% para países acima da média e 5% para países abaixo da média.

4 Modelagem

Com nossa a análise exploratória concluída, vamos dar início as estapas da modelagem utilizando o framework do tidymodels.

4.1 Amostragem

Fazendo a separação dos dados em treino e teste, estratificando pela variável resposta para a modelagem.

set.seed(32)

adult_split <- initial_split(adult, prop = 0.8, strata = resposta)

adult_train <- training(adult_split)
adult_test <- testing(adult_split)

4.2 Data Prep

Os tratamentos necessários observados na AED, que foi feita utilizando o pacote DataExplorer e a função AED_biv que gerei para entender o comportamento das variáveis com relação a variável resposta, serão armazenados utilizando o recipes para ser utilizado tanto para treinar os modelos como para testar posteriormente.

4.3 Cross-Validation

Especificando a validação cruzada:

set.seed(32)
adult_vfold <- vfold_cv(adult_train, v = 5, strata = resposta)
adult_vfold
#  5-fold cross-validation using stratification 

4.4 Modelos

Os modelos que serão ajustados:

  • Decision tree
  • Random Forest
  • Xgboost

Obs: Os valores dos hiperparâmetros foram obtidos a partir da tunagem e inseridos apenas para otimizar o tempo de renderização do script.

4.4.1 Decision tree

Especificando modelo:

 #1.069415e-09  8   19  Model04
adult_tree
Decision Tree Model Specification (classification)

Main Arguments:
  cost_complexity = 1.069415e-09
  tree_depth = 8
  min_n = 19

Computational engine: rpart 

Workflow para decision tree:

workflow_adult_tree
== Workflow ====================================================================
Preprocessor: Recipe
Model: decision_tree()

-- Preprocessor ----------------------------------------------------------------
7 Recipe Steps

* step_mutate()
* step_rm()
* step_string2factor()
* step_normalize()
* step_zv()
* step_novel()
* step_dummy()

-- Model -----------------------------------------------------------------------
Decision Tree Model Specification (classification)

Main Arguments:
  cost_complexity = 1.069415e-09
  tree_depth = 8
  min_n = 19

Computational engine: rpart 

Parâmentros:

hiperparams <- parameters(
 adult_tree
)
hiperparams
Collection of 3 parameters for tuning

              id  parameter type object class
 cost_complexity cost_complexity    nparam[+]
      tree_depth      tree_depth    nparam[+]
           min_n           min_n    nparam[+]

Grid:

Efetuando tunagem de hiperparâmetros:

Finalizando WF:

workflow_tree_final
== Workflow ====================================================================
Preprocessor: Recipe
Model: decision_tree()

-- Preprocessor ----------------------------------------------------------------
7 Recipe Steps

* step_mutate()
* step_rm()
* step_string2factor()
* step_normalize()
* step_zv()
* step_novel()
* step_dummy()

-- Model -----------------------------------------------------------------------
Decision Tree Model Specification (classification)

Main Arguments:
  cost_complexity = 1.069415e-09
  tree_depth = 8
  min_n = 19

Computational engine: rpart 

Verificando importância dos atributos:

Modelo final:

4.4.2 Random Forest

Especificando modelo:

# 23  1715    21
adult_rf
Random Forest Model Specification (classification)

Main Arguments:
  mtry = 23
  trees = 1715
  min_n = 21

Computational engine: randomForest 

Workflow para random forest:


workflow_adult_rf <- 
  adult_wf %>% 
  add_model(adult_rf)

Grid:

parameters(adult_rf)
Collection of 0 parameters for tuning

[1] id             parameter type object class  
<0 linhas> (ou row.names de comprimento 0)

Efetuando tunagem de hiperparâmetros:

set.seed(123)
rf_tune<- 
  workflow_adult_rf %>% 
  tune_grid(
    resamples = adult_vfold,
    grid = rf_grid,
    control = control_grid(save_pred = TRUE, verbose = T, allow_par = T),
    metrics = metric_set(roc_auc)
  )
i Fold1: recipe
v Fold1: recipe
i Fold1: model  1/10
v Fold1: model  1/10
i Fold1: model  1/10 (predictions)
i Fold1: model  2/10
v Fold1: model  2/10
i Fold1: model  2/10 (predictions)
i Fold1: model  3/10
v Fold1: model  3/10
i Fold1: model  3/10 (predictions)
i Fold1: model  4/10
v Fold1: model  4/10
i Fold1: model  4/10 (predictions)
i Fold1: model  5/10
v Fold1: model  5/10
i Fold1: model  5/10 (predictions)
i Fold1: model  6/10
v Fold1: model  6/10
i Fold1: model  6/10 (predictions)
i Fold1: model  7/10
v Fold1: model  7/10
i Fold1: model  7/10 (predictions)
i Fold1: model  8/10
v Fold1: model  8/10
i Fold1: model  8/10 (predictions)
i Fold1: model  9/10
v Fold1: model  9/10
i Fold1: model  9/10 (predictions)
i Fold1: model 10/10
v Fold1: model 10/10
i Fold1: model 10/10 (predictions)
i Fold2: recipe
v Fold2: recipe
i Fold2: model  1/10
v Fold2: model  1/10
i Fold2: model  1/10 (predictions)
i Fold2: model  2/10
v Fold2: model  2/10
i Fold2: model  2/10 (predictions)
i Fold2: model  3/10
v Fold2: model  3/10
i Fold2: model  3/10 (predictions)
i Fold2: model  4/10
v Fold2: model  4/10
i Fold2: model  4/10 (predictions)
i Fold2: model  5/10
v Fold2: model  5/10
i Fold2: model  5/10 (predictions)
i Fold2: model  6/10
v Fold2: model  6/10
i Fold2: model  6/10 (predictions)
i Fold2: model  7/10
v Fold2: model  7/10
i Fold2: model  7/10 (predictions)
i Fold2: model  8/10
v Fold2: model  8/10
i Fold2: model  8/10 (predictions)
i Fold2: model  9/10
v Fold2: model  9/10
i Fold2: model  9/10 (predictions)
i Fold2: model 10/10
v Fold2: model 10/10
i Fold2: model 10/10 (predictions)
i Fold3: recipe
v Fold3: recipe
i Fold3: model  1/10
v Fold3: model  1/10
i Fold3: model  1/10 (predictions)
i Fold3: model  2/10
v Fold3: model  2/10
i Fold3: model  2/10 (predictions)
i Fold3: model  3/10
v Fold3: model  3/10
i Fold3: model  3/10 (predictions)
i Fold3: model  4/10
v Fold3: model  4/10
i Fold3: model  4/10 (predictions)
i Fold3: model  5/10
v Fold3: model  5/10
i Fold3: model  5/10 (predictions)
i Fold3: model  6/10
v Fold3: model  6/10
i Fold3: model  6/10 (predictions)
i Fold3: model  7/10
v Fold3: model  7/10
i Fold3: model  7/10 (predictions)
i Fold3: model  8/10
v Fold3: model  8/10
i Fold3: model  8/10 (predictions)
i Fold3: model  9/10
v Fold3: model  9/10
i Fold3: model  9/10 (predictions)
i Fold3: model 10/10
v Fold3: model 10/10
i Fold3: model 10/10 (predictions)
i Fold4: recipe
v Fold4: recipe
i Fold4: model  1/10
v Fold4: model  1/10
i Fold4: model  1/10 (predictions)
i Fold4: model  2/10
v Fold4: model  2/10
i Fold4: model  2/10 (predictions)
i Fold4: model  3/10
v Fold4: model  3/10
i Fold4: model  3/10 (predictions)
i Fold4: model  4/10
v Fold4: model  4/10
i Fold4: model  4/10 (predictions)
i Fold4: model  5/10
v Fold4: model  5/10
i Fold4: model  5/10 (predictions)
i Fold4: model  6/10
v Fold4: model  6/10
i Fold4: model  6/10 (predictions)
i Fold4: model  7/10
v Fold4: model  7/10
i Fold4: model  7/10 (predictions)
i Fold4: model  8/10
v Fold4: model  8/10
i Fold4: model  8/10 (predictions)
i Fold4: model  9/10
v Fold4: model  9/10
i Fold4: model  9/10 (predictions)
i Fold4: model 10/10
v Fold4: model 10/10
i Fold4: model 10/10 (predictions)
i Fold5: recipe
v Fold5: recipe
i Fold5: model  1/10
v Fold5: model  1/10
i Fold5: model  1/10 (predictions)
i Fold5: model  2/10
v Fold5: model  2/10
i Fold5: model  2/10 (predictions)
i Fold5: model  3/10
v Fold5: model  3/10
i Fold5: model  3/10 (predictions)
i Fold5: model  4/10
v Fold5: model  4/10
i Fold5: model  4/10 (predictions)
i Fold5: model  5/10
v Fold5: model  5/10
i Fold5: model  5/10 (predictions)
i Fold5: model  6/10
v Fold5: model  6/10
i Fold5: model  6/10 (predictions)
i Fold5: model  7/10
v Fold5: model  7/10
i Fold5: model  7/10 (predictions)
i Fold5: model  8/10
v Fold5: model  8/10
i Fold5: model  8/10 (predictions)
i Fold5: model  9/10
v Fold5: model  9/10
i Fold5: model  9/10 (predictions)
i Fold5: model 10/10
v Fold5: model 10/10
i Fold5: model 10/10 (predictions)
rf_best_hiperparams <- select_best(rf_tune) 
Error in .get_tune_metric_names(x) : objeto 'rf_tune' não encontrado

Finalizando WF:

workflow_rf_final
== Workflow ====================================================================
Preprocessor: Recipe
Model: rand_forest()

-- Preprocessor ----------------------------------------------------------------
7 Recipe Steps

* step_mutate()
* step_rm()
* step_string2factor()
* step_normalize()
* step_zv()
* step_novel()
* step_dummy()

-- Model -----------------------------------------------------------------------
Random Forest Model Specification (classification)

Main Arguments:
  mtry = 23
  trees = 1715
  min_n = 21

Computational engine: randomForest 

Verificando importância dos atributos:

Modelo final:

4.4.3 Xgboost

Como o Xgboost possui muitos parâmetros, optei pora não tunnar os parâmetros loss_reduction e samples_size nesse primeiro momento. Sendo assim os valores default da enginee xgboost são atribuídos à esses parâmetros, loss_reduction = 0 e sample_size = 1.

adult_xgb
Boosted Tree Model Specification (classification)

Main Arguments:
  mtry = 34
  trees = 1309
  min_n = 5
  tree_depth = 10
  learn_rate = 0.0106

Computational engine: xgboost 

Workflow para Xgboost:

workflow_adult_xgb
== Workflow ====================================================================
Preprocessor: Recipe
Model: boost_tree()

-- Preprocessor ----------------------------------------------------------------
7 Recipe Steps

* step_mutate()
* step_rm()
* step_string2factor()
* step_normalize()
* step_zv()
* step_novel()
* step_dummy()

-- Model -----------------------------------------------------------------------
Boosted Tree Model Specification (classification)

Main Arguments:
  mtry = 34
  trees = 1309
  min_n = 5
  tree_depth = 10
  learn_rate = 0.0106

Computational engine: xgboost 

Grid:

xgb_grid <- parameters(adult_xgb) %>%
    finalize(bake(prep(adult_recipe),adult_train)) %>%
    grid_max_entropy(size = 20)
Erro: At least one parameter object is required.


Efetuando tunagem de hiperparâmetros:

library(doFuture)
all_cores <- parallel::detectCores(logical = FALSE) - 1

registerDoFuture()
cl <- makeCluster(all_cores)
plan(future::cluster, workers = cl)
getDoParWorkers()
[1] 3
# grid search
ini <- Sys.time()
xgb_tune <-
  workflow_adult_xgb %>%
    tune_grid(
        resamples = adult_vfold,
        grid = xgb_grid,
        control = control_grid(verbose = TRUE),
        metrics = metric_set(roc_auc)
    )
Warning in x :
  encerrando conexão não utilizada 6 (<-WNB027899SPO.ciandt.global:11937)
Warning in x :
  encerrando conexão não utilizada 5 (<-WNB027899SPO.ciandt.global:11937)
Warning in x :
  encerrando conexão não utilizada 4 (<-WNB027899SPO.ciandt.global:11937)
i Fold1: recipe
v Fold1: recipe
i Fold1: model  1/20
v Fold1: model  1/20
i Fold1: model  1/20 (predictions)
i Fold1: model  2/20
v Fold1: model  2/20
i Fold1: model  2/20 (predictions)
i Fold1: model  3/20
v Fold1: model  3/20
i Fold1: model  3/20 (predictions)
i Fold1: model  4/20
v Fold1: model  4/20
i Fold1: model  4/20 (predictions)
i Fold1: model  5/20
v Fold1: model  5/20
i Fold1: model  5/20 (predictions)
i Fold1: model  6/20
v Fold1: model  6/20
i Fold1: model  6/20 (predictions)
i Fold1: model  7/20
v Fold1: model  7/20
i Fold1: model  7/20 (predictions)
i Fold1: model  8/20
v Fold1: model  8/20
i Fold1: model  8/20 (predictions)
i Fold1: model  9/20
v Fold1: model  9/20
i Fold1: model  9/20 (predictions)
i Fold1: model 10/20
v Fold1: model 10/20
i Fold1: model 10/20 (predictions)
i Fold1: model 11/20
v Fold1: model 11/20
i Fold1: model 11/20 (predictions)
i Fold1: model 12/20
v Fold1: model 12/20
i Fold1: model 12/20 (predictions)
i Fold1: model 13/20
v Fold1: model 13/20
i Fold1: model 13/20 (predictions)
i Fold1: model 14/20
v Fold1: model 14/20
i Fold1: model 14/20 (predictions)
i Fold1: model 15/20
v Fold1: model 15/20
i Fold1: model 15/20 (predictions)
i Fold1: model 16/20
v Fold1: model 16/20
i Fold1: model 16/20 (predictions)
i Fold1: model 17/20
v Fold1: model 17/20
i Fold1: model 17/20 (predictions)
i Fold1: model 18/20
v Fold1: model 18/20
i Fold1: model 18/20 (predictions)
i Fold1: model 19/20
v Fold1: model 19/20
i Fold1: model 19/20 (predictions)
i Fold1: model 20/20
v Fold1: model 20/20
i Fold1: model 20/20 (predictions)
i Fold2: recipe
v Fold2: recipe
i Fold2: model  1/20
v Fold2: model  1/20
i Fold2: model  1/20 (predictions)
i Fold2: model  2/20
v Fold2: model  2/20
i Fold2: model  2/20 (predictions)
i Fold2: model  3/20
v Fold2: model  3/20
i Fold2: model  3/20 (predictions)
i Fold2: model  4/20
v Fold2: model  4/20
i Fold2: model  4/20 (predictions)
i Fold2: model  5/20
v Fold2: model  5/20
i Fold2: model  5/20 (predictions)
i Fold2: model  6/20
v Fold2: model  6/20
i Fold2: model  6/20 (predictions)
i Fold2: model  7/20
v Fold2: model  7/20
i Fold2: model  7/20 (predictions)
i Fold2: model  8/20
v Fold2: model  8/20
i Fold2: model  8/20 (predictions)
i Fold2: model  9/20
v Fold2: model  9/20
i Fold2: model  9/20 (predictions)
i Fold2: model 10/20
v Fold2: model 10/20
i Fold2: model 10/20 (predictions)
i Fold2: model 11/20
v Fold2: model 11/20
i Fold2: model 11/20 (predictions)
i Fold2: model 12/20
v Fold2: model 12/20
i Fold2: model 12/20 (predictions)
i Fold2: model 13/20
v Fold2: model 13/20
i Fold2: model 13/20 (predictions)
i Fold2: model 14/20
v Fold2: model 14/20
i Fold2: model 14/20 (predictions)
i Fold2: model 15/20
v Fold2: model 15/20
i Fold2: model 15/20 (predictions)
i Fold2: model 16/20
v Fold2: model 16/20
i Fold2: model 16/20 (predictions)
i Fold2: model 17/20
v Fold2: model 17/20
i Fold2: model 17/20 (predictions)
i Fold2: model 18/20
v Fold2: model 18/20
i Fold2: model 18/20 (predictions)
i Fold2: model 19/20
v Fold2: model 19/20
i Fold2: model 19/20 (predictions)
i Fold2: model 20/20
v Fold2: model 20/20
i Fold2: model 20/20 (predictions)
i Fold3: recipe
v Fold3: recipe
i Fold3: model  1/20
v Fold3: model  1/20
i Fold3: model  1/20 (predictions)
i Fold3: model  2/20
v Fold3: model  2/20
i Fold3: model  2/20 (predictions)
i Fold3: model  3/20
v Fold3: model  3/20
i Fold3: model  3/20 (predictions)
i Fold3: model  4/20
v Fold3: model  4/20
i Fold3: model  4/20 (predictions)
i Fold3: model  5/20
v Fold3: model  5/20
i Fold3: model  5/20 (predictions)
i Fold3: model  6/20
v Fold3: model  6/20
i Fold3: model  6/20 (predictions)
i Fold3: model  7/20
v Fold3: model  7/20
i Fold3: model  7/20 (predictions)
i Fold3: model  8/20
v Fold3: model  8/20
i Fold3: model  8/20 (predictions)
i Fold3: model  9/20
v Fold3: model  9/20
i Fold3: model  9/20 (predictions)
i Fold3: model 10/20
v Fold3: model 10/20
i Fold3: model 10/20 (predictions)
i Fold3: model 11/20
v Fold3: model 11/20
i Fold3: model 11/20 (predictions)
i Fold3: model 12/20
v Fold3: model 12/20
i Fold3: model 12/20 (predictions)
i Fold3: model 13/20
v Fold3: model 13/20
i Fold3: model 13/20 (predictions)
i Fold3: model 14/20
v Fold3: model 14/20
i Fold3: model 14/20 (predictions)
i Fold3: model 15/20
v Fold3: model 15/20
i Fold3: model 15/20 (predictions)
i Fold3: model 16/20
v Fold3: model 16/20
i Fold3: model 16/20 (predictions)
i Fold3: model 17/20
v Fold3: model 17/20
i Fold3: model 17/20 (predictions)
i Fold3: model 18/20
v Fold3: model 18/20
i Fold3: model 18/20 (predictions)
i Fold3: model 19/20
v Fold3: model 19/20
i Fold3: model 19/20 (predictions)
i Fold3: model 20/20
v Fold3: model 20/20
i Fold3: model 20/20 (predictions)
i Fold4: recipe
v Fold4: recipe
i Fold4: model  1/20
v Fold4: model  1/20
i Fold4: model  1/20 (predictions)
i Fold4: model  2/20
v Fold4: model  2/20
i Fold4: model  2/20 (predictions)
i Fold4: model  3/20
v Fold4: model  3/20
i Fold4: model  3/20 (predictions)
i Fold4: model  4/20
v Fold4: model  4/20
i Fold4: model  4/20 (predictions)
i Fold4: model  5/20
v Fold4: model  5/20
i Fold4: model  5/20 (predictions)
i Fold4: model  6/20
v Fold4: model  6/20
i Fold4: model  6/20 (predictions)
i Fold4: model  7/20
v Fold4: model  7/20
i Fold4: model  7/20 (predictions)
i Fold4: model  8/20
v Fold4: model  8/20
i Fold4: model  8/20 (predictions)
i Fold4: model  9/20
v Fold4: model  9/20
i Fold4: model  9/20 (predictions)
i Fold4: model 10/20
v Fold4: model 10/20
i Fold4: model 10/20 (predictions)
i Fold4: model 11/20
v Fold4: model 11/20
i Fold4: model 11/20 (predictions)
i Fold4: model 12/20
v Fold4: model 12/20
i Fold4: model 12/20 (predictions)
i Fold4: model 13/20
v Fold4: model 13/20
i Fold4: model 13/20 (predictions)
i Fold4: model 14/20
v Fold4: model 14/20
i Fold4: model 14/20 (predictions)
i Fold4: model 15/20
v Fold4: model 15/20
i Fold4: model 15/20 (predictions)
i Fold4: model 16/20
v Fold4: model 16/20
i Fold4: model 16/20 (predictions)
i Fold4: model 17/20
v Fold4: model 17/20
i Fold4: model 17/20 (predictions)
i Fold4: model 18/20
v Fold4: model 18/20
i Fold4: model 18/20 (predictions)
i Fold4: model 19/20
v Fold4: model 19/20
i Fold4: model 19/20 (predictions)
i Fold4: model 20/20
v Fold4: model 20/20
i Fold4: model 20/20 (predictions)
i Fold5: recipe
v Fold5: recipe
i Fold5: model  1/20
v Fold5: model  1/20
i Fold5: model  1/20 (predictions)
i Fold5: model  2/20
v Fold5: model  2/20
i Fold5: model  2/20 (predictions)
i Fold5: model  3/20
v Fold5: model  3/20
i Fold5: model  3/20 (predictions)
i Fold5: model  4/20
v Fold5: model  4/20
i Fold5: model  4/20 (predictions)
i Fold5: model  5/20
v Fold5: model  5/20
i Fold5: model  5/20 (predictions)
i Fold5: model  6/20
v Fold5: model  6/20
i Fold5: model  6/20 (predictions)
i Fold5: model  7/20
v Fold5: model  7/20
i Fold5: model  7/20 (predictions)
i Fold5: model  8/20
v Fold5: model  8/20
i Fold5: model  8/20 (predictions)
i Fold5: model  9/20
v Fold5: model  9/20
i Fold5: model  9/20 (predictions)
i Fold5: model 10/20
v Fold5: model 10/20
i Fold5: model 10/20 (predictions)
i Fold5: model 11/20
v Fold5: model 11/20
i Fold5: model 11/20 (predictions)
i Fold5: model 12/20
v Fold5: model 12/20
i Fold5: model 12/20 (predictions)
i Fold5: model 13/20
v Fold5: model 13/20
i Fold5: model 13/20 (predictions)
i Fold5: model 14/20
v Fold5: model 14/20
i Fold5: model 14/20 (predictions)
i Fold5: model 15/20
v Fold5: model 15/20
i Fold5: model 15/20 (predictions)
i Fold5: model 16/20
v Fold5: model 16/20
i Fold5: model 16/20 (predictions)
i Fold5: model 17/20
v Fold5: model 17/20
i Fold5: model 17/20 (predictions)
i Fold5: model 18/20
v Fold5: model 18/20
i Fold5: model 18/20 (predictions)
i Fold5: model 19/20
v Fold5: model 19/20
i Fold5: model 19/20 (predictions)
i Fold5: model 20/20
v Fold5: model 20/20
i Fold5: model 20/20 (predictions)
Sys.time()- ini #Time difference of 39.9844 mins(parallel)
Time difference of 48.61914 mins
foreach::registerDoSEQ()
xgb_best_hiperparams 
Erro: objeto 'xgb_best_hiperparams' não encontrado

Finalizando WF:

workflow_xgb_final
== Workflow ====================================================================
Preprocessor: Recipe
Model: boost_tree()

-- Preprocessor ----------------------------------------------------------------
7 Recipe Steps

* step_mutate()
* step_rm()
* step_string2factor()
* step_normalize()
* step_zv()
* step_novel()
* step_dummy()

-- Model -----------------------------------------------------------------------
Boosted Tree Model Specification (classification)

Main Arguments:
  mtry = 34
  trees = 1309
  min_n = 5
  tree_depth = 10
  learn_rate = 0.0106

Computational engine: xgboost 

Verificando importância dos atributos:

workflow_xgb_final %>%
  fit(adult_train) %>%
  pull_workflow_fit() %>%
  vip::vip(geom = "col")
`as.tibble()` is deprecated as of tibble 2.0.0.
Please use `as_tibble()` instead.
The signature and semantics have changed, see `?as_tibble`.
This warning is displayed once every 8 hours.
Call `lifecycle::last_warnings()` to see where this warning was generated.

Modelo final:

4.4.4 Xgboost2

Agora vamos inserir os valores identificados na tunagem para os parâmetros e efetuar o tuning para os parâmetros que sample_size e loss_reduction:

adult_xgb2
Boosted Tree Model Specification (classification)

Main Arguments:
  mtry = 34
  trees = 1309
  min_n = 5
  tree_depth = 10
  learn_rate = 0.0106445
  loss_reduction = 0.000127
  sample_size = 0.989

Computational engine: xgboost 

Workflow para Xgboost:

workflow_adult_xgb2
== Workflow ====================================================================
Preprocessor: Recipe
Model: boost_tree()

-- Preprocessor ----------------------------------------------------------------
7 Recipe Steps

* step_mutate()
* step_rm()
* step_string2factor()
* step_normalize()
* step_zv()
* step_novel()
* step_dummy()

-- Model -----------------------------------------------------------------------
Boosted Tree Model Specification (classification)

Main Arguments:
  mtry = 34
  trees = 1309
  min_n = 5
  tree_depth = 10
  learn_rate = 0.0106445
  loss_reduction = tune()
  sample_size = tune()

Computational engine: xgboost 

Grid:

Efetuando tunagem de hiperparâmetros:

getDoParWorkers()
[1] 3

Finalizando WF:

workflow_xgb_final2
== Workflow ====================================================================
Preprocessor: Recipe
Model: boost_tree()

-- Preprocessor ----------------------------------------------------------------
7 Recipe Steps

* step_mutate()
* step_rm()
* step_string2factor()
* step_normalize()
* step_zv()
* step_novel()
* step_dummy()

-- Model -----------------------------------------------------------------------
Boosted Tree Model Specification (classification)

Main Arguments:
  mtry = 34
  trees = 1309
  min_n = 5
  tree_depth = 10
  learn_rate = 0.0106445
  loss_reduction = 0.000127
  sample_size = 0.989

Computational engine: xgboost 

Verificando importância dos atributos:

Modelo final:

5 Comparando os modelos

Podemos ver a partir da curva roc que o xgboost obteve melhor perfomance que o random forest e a árvore de decisão. Interessante que o xgboost sem tunar os hiperparâmetros loss_reduction e sample_size se saiu discretamente melhor que o xgboost2 onde efetuamos o tuning desses dois hiperparâmetros. Sendo assim nosso modelo final será o xgb_final.

6 Scoragem para submeter resultado

Vamos então finalizar nosso modelo campeão e scorar a base de validação para efetuar a submissão:

xgboost_modelo_final
Boosted Tree Model Specification (classification)

Main Arguments:
  mtry = 34
  trees = 1309
  min_n = 5
  tree_depth = 10
  learn_rate = 0.0106

Computational engine: xgboost 

Matriz de confusão:

adult_val %>% 
  transmute(resposta = factor(resposta, levels = c(">50K", "<=50K")), 
            more_than_50k = ifelse(more_than_50k > 0.5, ">50K", "<=50K") %>% 
              factor(levels = c(">50K", "<=50K"))) %>% 
  table() %>% 
  caret::confusionMatrix()
Confusion Matrix and Statistics

        more_than_50k
resposta  >50K <=50K
   >50K   2505  1341
   <=50K   719 11716
                                          
               Accuracy : 0.8735          
                 95% CI : (0.8683, 0.8785)
    No Information Rate : 0.802           
    P-Value [Acc > NIR] : < 2.2e-16       
                                          
                  Kappa : 0.6286          
                                          
 Mcnemar's Test P-Value : < 2.2e-16       
                                          
            Sensitivity : 0.7770          
            Specificity : 0.8973          
         Pos Pred Value : 0.6513          
         Neg Pred Value : 0.9422          
             Prevalence : 0.1980          
         Detection Rate : 0.1539          
   Detection Prevalence : 0.2362          
      Balanced Accuracy : 0.8371          
                                          
       'Positive' Class : >50K            
                                          

Selecionando campos no formato da submissão:

submissao <- adult_val %>% select(id, more_than_50k)
write_csv(submissao, "submissao.csv")
LS0tDQp0aXRsZTogIkRlc2FmaW8gSW50cm8gTUwgLSBDdXJzby1SIg0KYXV0aG9yOiAiUmljYXJkbyBNYXR0b3MiDQpkYXRlOiAiMTIvMDcvMjAyMCINCm91dHB1dDoNCiAgaHRtbF9kb2N1bWVudDoNCiAgICB0b2M6IHllcw0KICAgIHRvY19mbG9hdDogeWVzDQogICAgbnVtYmVyX3NlY3Rpb25zOiB5ZXMNCiAgaHRtbF9ub3RlYm9vazoNCiAgICB0b2M6IHllcw0KICAgIHRvY19mbG9hdDogeWVzDQogICAgbnVtYmVyX3NlY3Rpb25zOiB5ZXMNCi0tLQ0KDQoNCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQ0KbGlicmFyeShyZWFkcikNCmxpYnJhcnkodGlkeW1vZGVscykNCmxpYnJhcnkoZ2dwbG90MikNCmxpYnJhcnkoc2tpbXIpDQpsaWJyYXJ5KFJDdXJsKQ0KbGlicmFyeShrYWJsZUV4dHJhKQ0KbGlicmFyeShncmlkRXh0cmEpDQpsaWJyYXJ5KGdsdWUpDQpsaWJyYXJ5KGZvcmNhdHMpDQpsaWJyYXJ5KERhdGFFeHBsb3JlcikNCmxpYnJhcnkoZTEwNzEpDQpgYGANCg0KIyBPYmpldGl2bw0KDQpPIG9iamV0aXZvIGRlc3NlIG5vdGVib29rIMOpIGVmZXR1YXIgdG9kbyBvIHByb2Nlc3NvIGRlIG1vZGVsYWdlbSBkYSBiYXNlIGRlIGRhZG9zIGBhZHVsdGAsIGRpc3BvbmliaWxpemFkYSBwYXJhIG8gZGVzYWZpbyBkbyBjdXJzbyBkZSBpbnRyb2R1w6fDo28gYW8gTWFjaGluZSBMZWFybmluZyBkYSBDdXJzby1SLCB1dGlsaXphbmRvIG8gZnJhbWV3b3JrIGB0aWR5bW9kZWxzYC4gT3Ugc2VqYSwgZXhwbG9yYXIsIHRyYXRhciwgcHJlcGFyYXIsIHR1bm5hciBlIGVzY29saGVyIG8gbW9kZWxvIHF1ZSBtZWxob3Igc2UgYWp1c3RhIGFvcyBkYWRvcyBkaXNwb25pYmlsaXphZG9zLg0KDQoNCiMgTGVpdHVyYSBkYSBiYXNlDQoNCiMjIEluZm9ybWHDp8O1ZXMgcHJlbGltaW5hcmVzDQoNCmBgYHtyfQ0KYWR1bHQgPC0gcmVhZF9yZHMoImFkdWx0LnJkcyIpDQoNCiMgaGVhZChhZHVsdCkgDQoNCiMgZ2xpbXBzZShhZHVsdCkNCnNraW0oYWR1bHQpDQoNCmBgYA0KDQoNCg0KPGJyPiBBcyB2YXJpw6F2ZWlzIHBhcmVjZW0gZXN0YXIgY29tIGZvcm1hdG9zIGNvcnJldG9zLiBQb250byBkZSBhdGVuw6fDo28gcGFyYSBhcyB2YXJpw6F2ZWlzIGB3b2tjbGFzc2AsIGBvY2N1cGF0aW9uYCBlIGBuYXRpdmVfY291bnRyeWAsIHF1ZSBhcHJlc2VudGFtIHZhbG9yZXMgbWlzc2luZy4gIDwvYnI+DQoNCg0KIyBBRUQgDQoNCjxicj4gQWdvcmEgdmFtb3MgYW5hbGlzYXIgbyBjb21wb3J0YW1lbnRvIGRhcyB2YXJpw6F2ZWlzIHBhcmEgZGVmaW5pcm1vcyBjb21vIHRyYXRhciBvcyBub3Nzb3MgZGFkb3MgcGFyYSBvIG1vZGVsby4gPC9icj4NCg0KIyMgUGFydGUgMSB7LnRhYnNldH0NCg0KYGBge3IscmVzdWx0cz0nYXNpcycsIGVjaG89VFJVRSwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0NCg0KDQojIERhdGFFeHBsb3Jlcjo6Y3JlYXRlX3JlcG9ydChhZHVsdCkNCg0KZGV2dG9vbHM6OnNvdXJjZV91cmwoImh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9yaWNhcmRvbWF0dG9zMDUvZnVuY3Rpb25zL21hc3Rlci9mdW5jdGlvbl9BRURfYml2YXJpYWRhLlIiKQ0KIyANCiMgDQphZHVsdDIgPC0gYWR1bHQgJT4lDQogICAgICAgICAgICBzZWxlY3QoLWlkKSAlPiUgDQogICAgICAgICAgICBtdXRhdGUocmVzcG9zdGEgPSBpZl9lbHNlKHJlc3Bvc3RhID09ICI+NTBLIiwgMSwgMCkpDQojIA0KIyANCg0KIyBuYW1lcyhhZHVsdDIpDQpmb3IgKGkgaW4gMToobGVuZ3RoKGFkdWx0MiktMSkgKSB7DQogIA0KICBkZiA8LSBhZHVsdDJbLGMoaSwxNSldDQogIGNhdCgiIyMjICIsbmFtZXMoZGZbLDFdKSwiXG4iKSANCiAgcHJpbnQoQUVEX2JpdihkZixnbHVlKCJyZXNwb3N0YSIpLCJQcmUiKSkNCiAgY2F0KCdcblxuJykNCn0NCg0KDQoNCmBgYA0KDQoNCiMjIHstfQ0KDQpPYnNlcnZhw6fDtWVzOg0KDQoqIGBlZHVjYXRpb25gIDogw6kgcG9zc8OtdmVsIHZpc3VhbGl6YXIgcXVlIHF1YW50byBtYWlvciBvIGdyYXUgZGUgZXNjb2xhcmlkYWRlLCBtYWlvciBhIHByb3BvcsOnw6NvIGRlIHBlc3NvYXMgY29tIHNhbGFyaW9zIGFjaW1hIGRlIDUway4gRSBxdWUgYXMgY2F0ZWdvcmlhcyBhYmFpeG8gZGUgSFMtZ3JhZCwgYDF0aC00dGhgIGF0w6kgYDEydGhgYWzDqW0gZGUgc2VyZW0gcG91Y28gcmVwcmVzZW50YXRpdmFzLCBwb3NzdWVtIGJhaXhhIHByb3BvcsOnw6NvLCB2YW1vcyBlbnTDo28gY3JpYXIgdW1hIGNhdGVnb3JpYSB1bWEgbm92YSBjb25zb2xpZGFuZG8gZWxhcyBgSFMtbm90LWdyYWRgLg0KDQoqIGBtYXJpdGFsX3N0YXR1c2AgOiBhcXVpIGlyZW1vcyBhZ3J1cGFyIG9zIGNhbXBvcyBgTWFycmllZC1BRi1zcG91c2VgIGUgYE1hcnJpZWQtY2l2LXNwb3VzZWAsIGNyaWFuZG8gYSBjYXRlZ29yaWEgYE1hcnJpZWRgLCBiYXNlYWRvIG5hIHNpbWlsYXJpZGFkZSBlbnRyZSBlbGFzIGNvbSByZWxhw6fDo28gYSB2YXJpw6F2ZWwgcmVzcG9zdGEgZSBjb25zaWRlcmFuZG8gYSBkZXNjcmnDp8OjbyBkZWxhcy4NCg0KKiBgbmF0aXZlX2NvdW50cnlgIDogw4kgdW0gY2FtcG8gY29tIHBvdWNhIHZhcmlhYmlsaWRhZGUsIG9uZGUgYHIgKGFkdWx0ICU+JSBzZWxlY3QobmF0aXZlX2NvdW50cnkpICU+JSBmaWx0ZXIobmF0aXZlX2NvdW50cnkgPT0gIlVuaXRlZC1TdGF0ZXMiKSAlPiUgY291bnQoKSAvIGNvdW50KGFkdWx0KSkgJT4lIGFzLm51bWVyaWMoKSAlPiUgcGVyY2VudCgpYCBkb3MgZGFkb3MgZXN0w6NvIGF0cmlidcOtZG9zIGNvbW8gIkVzdGFkb3MgVW5pZG9zIi4gU2VuZG8gYXNzaW0sIHBvZGVyaWEgY29uc2lkZXJhciBhcGVuYXMgRXN0YWRvcyBVbmlkb3MgZSBhZ3J1cGFyIG8gcmVzdGFudGUgY29tbyBvdXRyb3MsIG1hcyB2YW1vcyBtYW50ZXIgbyBtw6F4aW1vIGRlIGluZm9ybWHDp8OjbyBlIHJlZHV6aXIgYXMgY2F0ZWdvcmlhcyBwYXJhIDMsIGFncnVwYW5kbyB0b2RvcyBvcyBwYcOtc2VzIHF1ZSBvYnRpdmVyYW0gcHJvcG9yw6fDo28gbWFpb3IgcXVlIGEgbcOpZGlhLCBtYW50ZXIgbyB2YWxvciBtYWlzIHJlcHJlc2VudGF0aXZvIGUgdW1hIGNhdGVnb3JpYSBjb20gb3MgcGHDrXNlcyBhYmFpeG8gZGEgbcOpZGlhLg0KDQoqIGByZWxhdGlvbnNoaXBgIDogY2FtcG8gY29udMOpbSBvcyBjYW1wb3MgYGh1c2JhbmRgIGUgYHdpZmVgLCBhcGFyZW50ZW1lbnRlIHBvZGVyaWFtb3MgYWdydXBhLWxvcywgdmFtb3MgYW5hbGlzYXIgbWFpcyBhZnVuZG8uDQoNCiogYGNhcGl0YWxfbG9zc2AgZSBgY2FwaXRhbF9nYWluYCA6IEFwYXJlbnRlbWVudGUgdGFudG8gcXVlbSBnYW5oYSBxdWFudG8gcXVlbSBwZXJkZSBhbGd1bSB2YWxvciBhcHJlc2VudGFtIG1haW9yZXMgcHJvYmFiaWxpZGFkZXMgZGUgdGVyIHNhbGFyaW8gPjUway4gVmFtb3MgZW50w6NvIGF2YWxpYXIgYSBjb3JyZWxhw6fDo28gZW50cmUgZWxhcy4NCg0KKiBgd29ya2NsYXNzYCA6IENhdGVnb3JpYXMgY29tIGJhaXhhIHJlcHJlc2VudGF0aXZpZGFkZSBjb21vIGBOZXZlci13b3JrZWRgZSBgV2l0aG91dC1wYXlgIG7Do28gcG9zc3VlbSBjbGFzc2lmaWNhw6fDo28gY29tIGEgcmVzcG9zdGEgZGUgaW50ZXJlc3NlICI+NTBrIiwgdmFtb3MgZGFyIHVtIHpvb20gbmVzc2EgdmFyacOhdmVsIGUgYW5hbGlzYXIgb3MgTkEncyBxdWUgaWRlbnRpZmljYW1vcyB0YW1iw6ltLg0KDQojIyBQYXJ0ZSAyIHsudGFic2V0fQ0KDQojIyMgb2NjdXBhdGlvbg0KDQpgYGB7cn0NCg0KZ2dwbG90KGFkdWx0LCBhZXMoeCA9IG9jY3VwYXRpb24sIGZpbGwgPSByZXNwb3N0YSkpICsgDQogIGdlb21fYmFyKHBvc2l0aW9uPSJmaWxsIikgKyANCiAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA5MCkpICsgDQogIGdndGl0bGUoIm9jY3VwYXRpb24iKQ0KDQpgYGANCg0KPGJyPiDDiSBwb3Nzw612ZWwgdmVyIHF1ZSBuw6NvIGZhcmlhIHNlbnRpZG8gYXRyaWJ1aXIgb3MgTkFzIGRlIGZvcm1hIG1vZGFsLCB1bWEgdmV6IHF1ZSBub3NzbyBvYmpldGl2byDDqSBvYnRlciBvIG1haW9yIHBvZGVyIHByZWRpdGl2byBwb3Nzw612ZWwsIGxvZ28sIG7Do28gcXVlcmVtb3MgcGVyZGVyIGluZm9ybWHDp8Ojby4gU2VuZG8gYXNzaW0sIG7Do28gdmFtb3MgZGlsdWlyIG9zIE5BcyBuYSBjYXRlZ29yaWEgY29tIG1haW9yIHJlcHJlc2VudGF0aXZpZGFkZSBgUHJvZi1zcGVjaWFsdHlgLCB2YW1vcyBhdHJpYnVpciDDoCB1bWEgY2F0ZWdvcmlhIGNvbSBwcm9wb3LDp8O1ZXMgc2ltaWxhcmVzIGUgcXVlIHBvc3N1aSB1bWEgYm9hIHJlcHJlc2VudGF0aXZpZGFkZSwgYEZhcm1pbmctZmlzaGluZ2AuIDwvYnI+DQoNCg0KIyMjIHJlbGF0aW9uc2hpcA0KYGBge3J9DQoNCmdncGxvdChhZHVsdCwgYWVzKHggPSByZWxhdGlvbnNoaXApKSArDQogIGdlb21fYmFyKCkgKw0KICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDkwKSkgKyANCiAgZ2d0aXRsZSgicmVsYXRpb25zaGlwIikNCg0KZ2dwbG90KGFkdWx0LCBhZXMoeCA9IHJlbGF0aW9uc2hpcCwgZmlsbCA9IHJlc3Bvc3RhKSkgKyANCiAgZ2VvbV9iYXIocG9zaXRpb249ImZpbGwiKSArIA0KICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDkwKSkgKyANCiAgZ2d0aXRsZSgicmVsYXRpb25zaGlwIikNCg0KZ2dwbG90KGFkdWx0LCBhZXMoeCA9IHJlbGF0aW9uc2hpcCwgZmlsbCA9IHNleCkpICsgDQogIGdlb21fYmFyKHBvc2l0aW9uPSJmaWxsIikgKyANCiAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA5MCkpICsgDQogIGdndGl0bGUoInJlbGF0aW9uc2hpcCIpDQoNCmBgYA0KDQpWYW1vcyBlbnTDo28gYmFsYW5jZWFyIG8gZ8OqbmVybyBhZ3J1cGFuZG8gYXMgY2F0ZWdvcmlhcyBXaWZlIGUgSHVzYmFuZCwgY3JpYW5kbyBhIGNhdGVnb3JpYSBgTWFycmllZGAuDQoNCiMjIyBDYXBpdGFsIEdhaW4gYW5kIExvc3MNCg0KDQpgYGB7ciwgZWNobyA9IFRSVUV9DQoNCmdncGxvdChhZHVsdCwgYWVzKHg9IGNhcGl0YWxfZ2FpbiwgeT0gY2FwaXRhbF9sb3NzKSkgKw0KICBnZW9tX3BvaW50KCkNCmBgYA0KDQoNCg0KYGBge3J9DQpzdW0oYWR1bHQkY2FwaXRhbF9sb3NzID4gMCAmIGFkdWx0JGNhcGl0YWxfZ2FpbiA+IDApDQpgYGANClNlbmRvIGFzc2ltLCBwb2RlbW9zIHNvbWEtbGFzIGUgY3JpYXIgYSB2YXJpw6F2ZWwgYGNhcGl0YWxfdG90YWxgIHNlbSBtZWRvIGRlIHBlcmRlciBpbmZvcm1hw6fDo28uDQoNCiMjIyBXb3JjbGFzcw0KDQpgYGB7cn0NCg0KZ2dwbG90KGFkdWx0LCBhZXMoeCA9IHdvcmtjbGFzcykpICsNCiAgZ2VvbV9iYXIoKSArDQogIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KGFuZ2xlID0gOTApKSArIA0KICBnZ3RpdGxlKCJXb3JrY2xhc3MiKQ0KDQpnZ3Bsb3QoYWR1bHQsIGFlcyh4ID0gd29ya2NsYXNzLCBmaWxsID0gcmVzcG9zdGEpKSArIA0KICBnZW9tX2Jhcihwb3NpdGlvbj0iZmlsbCIpICsgDQogIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KGFuZ2xlID0gOTApKSArIA0KICBnZ3RpdGxlKCJXb3JrY2xhc3MiKQ0KDQpgYGANCg0KUGVsbyB2aXN0byBhIGNhdGdvcmlhIE5BIHBvc3N1aSByZWxhw6fDo28gY29tIGEgdmFyacOhdmVsIHJlc3Bvc3RhIGRpc3RpbnRhIGRlIHRvZGFzIGFzIG91dHJhcyBjYXRlZ29yaWFzLCB2YW1vcyBlbnTDo28gZ2VyYXIgdW1hIG5vdmEgY2F0ZWdvcmlhIGBub3QtaWRlbnRpZnlgIHBhcmEgYXRyaWJ1aXIgb3MgdmFsb3JlcyBOQS4NCg0KDQojIyMgbmF0aXZlX2NvdW50cnkNCg0KYGBge3J9DQoNCm1lZCA8LSAoYWR1bHQgJT4lIA0KICAgICAgICAgIHNlbGVjdChyZXNwb3N0YSkgJT4lDQogICAgICAgICAgZmlsdGVyKHJlc3Bvc3RhID09ICI+NTBLIikgJT4lIA0KICAgICAgICAgIGNvdW50KCkgJT4lIA0KICAgICAgICAgIGFzLm51bWVyaWMoKSkvbnJvdyhhZHVsdCkNCiAgICAgICAgIA0KDQp0Yl9jb3VudHJ5PC0gYWR1bHQgJT4lIA0KICAgICAgICAgICAgICAgIHNlbGVjdChuYXRpdmVfY291bnRyeSwgcmVzcG9zdGEpICU+JSANCiAgICAgICAgICAgICAgICBncm91cF9ieShuYXRpdmVfY291bnRyeSkgJT4lIA0KICAgICAgICAgICAgICAgIGNvdW50KHJlc3Bvc3RhKSAlPiUgDQogICAgICAgICAgICAgICAgbXV0YXRlKHByb3AgPSBwcm9wLnRhYmxlKG4pKSAlPiUgDQogICAgICAgICAgICAgICAgZmlsdGVyKHJlc3Bvc3RhID09ICI+NTBLIikgJT4lIA0KICAgICAgICAgICAgICAgIG11dGF0ZSggY2xhc3MgPSBjYXNlX3doZW4oIG5hdGl2ZV9jb3VudHJ5ID09ICJVbml0ZWQtU3RhdGVzIiB+ICJVbml0ZWQtU3RhdGVzIiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcm9wID4gbWVkIH4gIj5tZWFuIiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwcm9wIDw9IG1lZCB+ICI8PW1lYW4iICkgICkNCg0KdGJfY291bnRyeSAlPiUgDQogIHNlbGVjdChuYXRpdmVfY291bnRyeSxjbGFzcykgJT4lIA0KICBncm91cF9ieShjbGFzcykgJT4lIA0KICBjb3VudCgpDQoNCg0KDQpgYGANCg0KDQpGaWNhbW9zIGVudMOjbyBjb20gMjEgcGHDrXNlcyBjb20gcHJvcG9yw6fDtWVzIGFiYWl4byBkYSBtw6lkYSwgMTggYWNpbWEgZSAiVW5pdGVkLVN0YXRlcyIgY29tbyBhcyAzIGNhdGVnb3JpYXMgcmVzdGFudGVzLg0KDQoNCmBgYHtyLCBlY2hvPUZBTFNFfQ0KDQojIHRiX2NvdW50cnkgJT4lDQojICAgICAgICAgZmlsdGVyKGNsYXNzID09ICI8PW1lYW4iKSAlPiUNCiMgICAgICAgICBzZWxlY3QobmF0aXZlX2NvdW50cnkpICU+JQ0KIyAgICAgICAgIGFzLmZhY3RvcigpDQoNCg0KYWR1bHQyPC0gYWR1bHQyICU+JQ0KICAgIG11dGF0ZShjbGFzc19jb3VudHJ5ID0gY2FzZV93aGVuKG5hdGl2ZV9jb3VudHJ5ICVpbiUgYygiQ2FtYm9kaWEiLCAiQ2FuYWRhIiwgIkNoaW5hIiwgIkN1YmEiLCAiRW5nbGFuZCIsICJGcmFuY2UiLCAiR2VybWFueSIsICJHcmVlY2UiLCAiSG9uZyIsICJJbmRpYSIsICJJcmFuIiwgIkl0YWx5IiwgIkphcGFuIiwgIlBoaWxpcHBpbmVzIiwgIlNjb3RsYW5kIiwgIlRhaXdhbiIsICJZdWdvc2xhdmlhIiwgTkEpICB+ICI+bWVhbiIsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5hdGl2ZV9jb3VudHJ5ID09ICJVbml0ZWQtU3RhdGVzIiB+ICJVbml0ZWQtU3RhdGVzIiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgVFJVRSB+ICI8PW1lYW4iKSApICAgICANCg0KIyBhZHVsdDIgJT4lIA0KIyAgIGZpbHRlcihjbGFzcyA9PSAiPm1lYW4iKSAlPiUgDQojICAgc2VsZWN0KG5hdGl2ZV9jb3VudHJ5KSAlPiUgDQojICAgZ3JvdXBfYnkobmF0aXZlX2NvdW50cnkpICU+JSANCiMgICBjb3VudCgpDQoNCmdncGxvdChhZHVsdDIsIGFlcyh4ID0gY2xhc3NfY291bnRyeSkpICsNCiAgZ2VvbV9iYXIoYWVzKHkgPSAoLi5jb3VudC4uKS9zdW0oLi5jb3VudC4uKSkpICsNCiAgZ2VvbV90ZXh0KHN0YXQgPSAiY291bnQiLCANCiAgICAgICAgICAgIGFlcyhsYWJlbCA9IHJvdW5kKCguLmNvdW50Li4pL3N1bSguLmNvdW50Li4pLCAyKSwgeSA9IC4ucHJvcC4uICsgMC4wMikpKw0KICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDkwKSkgKyANCiAgc2NhbGVfeV9jb250aW51b3VzKGxhYmVscz1wZXJjZW50KSsgeWxhYigicHJvcCIpKw0KICBnZ3RpdGxlKCJjbGFzc19jb3VudHJ5IikNCg0KZ2dwbG90KGFkdWx0MiwgYWVzKHggPSBjbGFzc19jb3VudHJ5LCBmaWxsID0gYXMuZmFjdG9yKHJlc3Bvc3RhKSApKSArIA0KICBnZW9tX2Jhcihwb3NpdGlvbj0iZmlsbCIpICsgDQogIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KGFuZ2xlID0gOTApKSArIA0KICBzY2FsZV95X2NvbnRpbnVvdXMobGFiZWxzPXBlcmNlbnQpKw0KICBnZ3RpdGxlKCJjbGFzc19jb3VudHJ5IikNCiAgICANCiAgICANCmBgYA0KDQo8YnI+QSBkaXN0cmlidWnDp8OjbyBmaWNvdSBjb20gNSUgcGFyYSBwYcOtc2VzIGFjaW1hIGRhIG3DqWRpYSBlIDUlIHBhcmEgcGHDrXNlcyBhYmFpeG8gZGEgbcOpZGlhLjwvYnI+DQoNCiMgTW9kZWxhZ2VtDQoNCkNvbSBub3NzYSBhIGFuw6FsaXNlIGV4cGxvcmF0w7NyaWEgY29uY2x1w61kYSwgdmFtb3MgZGFyIGluw61jaW8gYXMgZXN0YXBhcyBkYSBtb2RlbGFnZW0gdXRpbGl6YW5kbyBvIGZyYW1ld29yayBkbyBgdGlkeW1vZGVsc2AuDQoNCiMjIEFtb3N0cmFnZW0NCg0KRmF6ZW5kbyBhIHNlcGFyYcOnw6NvIGRvcyBkYWRvcyBlbSB0cmVpbm8gZSB0ZXN0ZSwgZXN0cmF0aWZpY2FuZG8gcGVsYSB2YXJpw6F2ZWwgcmVzcG9zdGEgcGFyYSBhIG1vZGVsYWdlbS4NCg0KYGBge3IsIGVjaG89VFJVRSwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0NCnNldC5zZWVkKDMyKQ0KDQphZHVsdF9zcGxpdCA8LSBpbml0aWFsX3NwbGl0KGFkdWx0LCBwcm9wID0gMC44LCBzdHJhdGEgPSByZXNwb3N0YSkNCg0KYWR1bHRfdHJhaW4gPC0gdHJhaW5pbmcoYWR1bHRfc3BsaXQpDQphZHVsdF90ZXN0IDwtIHRlc3RpbmcoYWR1bHRfc3BsaXQpDQoNCmBgYA0KDQoNCiMjIERhdGEgUHJlcA0KDQpPcyB0cmF0YW1lbnRvcyBuZWNlc3PDoXJpb3Mgb2JzZXJ2YWRvcyBuYSBBRUQsIHF1ZSBmb2kgZmVpdGEgdXRpbGl6YW5kbyBvIHBhY290ZSBgRGF0YUV4cGxvcmVyYCBlIGEgZnVuw6fDo28gW2BBRURfYml2YF0oaHR0cHM6Ly9naXRodWIuY29tL3JpY2FyZG9tYXR0b3MwNS9mdW5jdGlvbnMvYmxvYi9tYXN0ZXIvZnVuY3Rpb25fQUVEX2JpdmFyaWFkYS5SKSBxdWUgZ2VyZWkgcGFyYSBlbnRlbmRlciBvIGNvbXBvcnRhbWVudG8gZGFzIHZhcmnDoXZlaXMgY29tIHJlbGHDp8OjbyBhIHZhcmnDoXZlbCByZXNwb3N0YSwgc2Vyw6NvIGFybWF6ZW5hZG9zIHV0aWxpemFuZG8gbyByZWNpcGVzIHBhcmEgc2VyIHV0aWxpemFkbyB0YW50byBwYXJhIHRyZWluYXIgb3MgbW9kZWxvcyBjb21vIHBhcmEgdGVzdGFyIHBvc3Rlcmlvcm1lbnRlLg0KDQoNCmBgYHtyfQ0KDQphZHVsdF9yZWNpcGUgPC0gDQogIHJlY2lwZShyZXNwb3N0YSB+IC4sIGRhdGEgPSBhZHVsdF90cmFpbikgJT4lIA0KICBzdGVwX211dGF0ZSgNCiAgICANCiAgICBvY2N1cGF0aW9uID0gY2FzZV93aGVuKA0KICAgICAgaXMubmEob2NjdXBhdGlvbikgfiAiRmFybWluZy1maXNoaW5nIiwNCiAgICAgIFRSVUUgfiBhcy5jaGFyYWN0ZXIob2NjdXBhdGlvbikpLA0KICAgIA0KICAgIHdvcmtjbGFzcyA9IGNhc2Vfd2hlbigNCiAgICAgIGlzLm5hKHdvcmtjbGFzcykgfiAiTm90LWlkZW50aWZ5IiwNCiAgICAgIFRSVUUgfiBhcy5jaGFyYWN0ZXIod29ya2NsYXNzKSksDQogICAgDQogICAgY2xhc3NfY291bnRyeSA9IGNhc2Vfd2hlbihuYXRpdmVfY291bnRyeSAlaW4lIGMoIkNhbWJvZGlhIiwgIkNhbmFkYSIsICJDaGluYSIsICJDdWJhIiwgIkVuZ2xhbmQiLCAiRnJhbmNlIiwgIkdlcm1hbnkiLCAiR3JlZWNlIiwgIkhvbmciLCAiSW5kaWEiLCAiSXJhbiIsICJJdGFseSIsICJKYXBhbiIsICJQaGlsaXBwaW5lcyIsICJTY290bGFuZCIsICJUYWl3YW4iLCAiWXVnb3NsYXZpYSIsIE5BKSB+ICJncmVhdGVyX21lYW4iLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuYXRpdmVfY291bnRyeSA9PSAiVW5pdGVkLVN0YXRlcyIgfiAiVW5pdGVkLVN0YXRlcyIsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgIFRSVUUgfiAic21hbGxlcl9tZWFuIikNCiAgICAsDQogICAgDQogICAgY2FwaXRhbF90b3RhbCA9IGNhcGl0YWxfZ2FpbiArIGNhcGl0YWxfbG9zcw0KICAgICwgDQogICAgDQogICAgbWFyaXRhbF9zdGF0dXMgPSBjYXNlX3doZW4oDQogICAgICBtYXJpdGFsX3N0YXR1cyAlaW4lIGMoIk1hcnJpZWQtQUYtc3BvdXNlIiAsICJNYXJyaWVkLWNpdi1zcG91c2UiKSB+ICJNYXJyaWVkIiwNCiAgICAgIFRSVUUgfiBhcy5jaGFyYWN0ZXIobWFyaXRhbF9zdGF0dXMpKQ0KICAgICwNCiAgICANCiAgICBlZHVjYXRpb24gPSBjYXNlX3doZW4oZWR1Y2F0aW9uICVpbiUgYygiMXN0LTR0aCIsICI1dGgtNnRoIiwgIjd0aC04dGgiLCAiOXRoIiwgIjEwdGgiLCAiMTF0aCIsICIxMnRoIikgfiAiSFMtbm90LWdyYWQiLA0KICAgICAgICAgICAgICAgICAgICAgICAgICBUUlVFIH4gYXMuY2hhcmFjdGVyKGVkdWNhdGlvbikpDQogICAgLA0KICAgIA0KICAgIHJlbGF0aW9uc2hpcCA9IGNhc2Vfd2hlbiggIHJlbGF0aW9uc2hpcCAlaW4lIGMoIkh1c2JhbmQiLCJXaWZlIikgfiAiTWFycmllZCIsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgVFJVRSB+IGFzLmNoYXJhY3RlcihyZWxhdGlvbnNoaXApKQ0KICAgIA0KICApICU+JSANCiAgc3RlcF9ybShpZCwgY2FwaXRhbF9nYWluLCBjYXBpdGFsX2xvc3MsIG5hdGl2ZV9jb3VudHJ5KSU+JSANCiAgc3RlcF9zdHJpbmcyZmFjdG9yKGFsbF9ub21pbmFsKCkpICU+JQ0KICBzdGVwX25vcm1hbGl6ZShhbGxfbnVtZXJpYygpKSAlPiUgDQogIHN0ZXBfenYoYWxsX3ByZWRpY3RvcnMoKSkgJT4lDQogIHN0ZXBfbm92ZWwoYWxsX25vbWluYWwoKSwgLWFsbF9vdXRjb21lcygpKSAlPiUgDQogIHN0ZXBfZHVtbXkoYWxsX25vbWluYWwoKSwgLWFsbF9vdXRjb21lcygpKQ0KDQogIyBiYWtlKHByZXAoYWR1bHRfcmVjaXBlKSwgYWR1bHRfdHJhaW4pDQoNCg0KYWR1bHRfd2YgPC0gDQogIHdvcmtmbG93KCkgJT4lIA0KICBhZGRfcmVjaXBlKGFkdWx0X3JlY2lwZSkNCmBgYA0KDQoNCg0KIyMgQ3Jvc3MtVmFsaWRhdGlvbg0KDQpFc3BlY2lmaWNhbmRvIGEgdmFsaWRhw6fDo28gY3J1emFkYToNCg0KYGBge3J9DQpzZXQuc2VlZCgzMikNCmFkdWx0X3Zmb2xkIDwtIHZmb2xkX2N2KGFkdWx0X3RyYWluLCB2ID0gNSwgc3RyYXRhID0gcmVzcG9zdGEpDQphZHVsdF92Zm9sZA0KYGBgDQoNCiMjIE1vZGVsb3Mgey50YWJzZXR9DQoNCk9zIG1vZGVsb3MgcXVlIHNlcsOjbyBhanVzdGFkb3M6DQoNCiAgKiBEZWNpc2lvbiB0cmVlDQogICogUmFuZG9tIEZvcmVzdA0KICAqIFhnYm9vc3QNCiAgDQpPYnM6IE9zIHZhbG9yZXMgZG9zIGhpcGVycGFyw6JtZXRyb3MgZm9yYW0gb2J0aWRvcyBhIHBhcnRpciBkYSB0dW5hZ2VtIGUgaW5zZXJpZG9zIGFwZW5hcyBwYXJhIG90aW1pemFyIG8gdGVtcG8gZGUgcmVuZGVyaXphw6fDo28gZG8gc2NyaXB0Lg0KICANCiMjIyBEZWNpc2lvbiB0cmVlDQoNCkVzcGVjaWZpY2FuZG8gbW9kZWxvOg0KDQpgYGB7cn0NCmFkdWx0X3RyZWUgPC0gDQogIGRlY2lzaW9uX3RyZWUoDQogICAgbWluX24gPSAxOSwNCiAgICBjb3N0X2NvbXBsZXhpdHkgPSAxLjA2OTQxNWUtMDksIA0KICAgIHRyZWVfZGVwdGggPSA4KSAlPiUNCiAgc2V0X21vZGUoImNsYXNzaWZpY2F0aW9uIikgJT4lDQogIHNldF9lbmdpbmUoInJwYXJ0IikNCiAjMS4wNjk0MTVlLTA5CTgJMTkJTW9kZWwwNA0KYWR1bHRfdHJlZQ0KYGBgDQpXb3JrZmxvdyBwYXJhIGRlY2lzaW9uIHRyZWU6DQoNCmBgYHtyfQ0KDQp3b3JrZmxvd19hZHVsdF90cmVlIDwtIA0KICBhZHVsdF93ZiAlPiUgDQogIGFkZF9tb2RlbChhZHVsdF90cmVlKQ0KDQoNCmBgYA0KDQpQYXLDom1lbnRyb3M6DQoNCmBgYHtyfQ0KIyBoaXBlcnBhcmFtcyA8LSBwYXJhbWV0ZXJzKA0KIyAgYWR1bHRfdHJlZQ0KIyApDQojIGhpcGVycGFyYW1zDQpgYGANCg0KR3JpZDoNCg0KYGBge3J9DQojIHNldC5zZWVkKDMyKQ0KIyB0cmVlX2dyaWQgPC0gZ3JpZF9tYXhfZW50cm9weShoaXBlcnBhcmFtcywgc2l6ZSA9IDEwKQ0KIyB0cmVlX2dyaWQNCg0KYGBgDQoNCg0KRWZldHVhbmRvIHR1bmFnZW0gZGUgaGlwZXJwYXLDom1ldHJvczoNCg0KYGBge3IsIGVjaG89IFRSVUUsIHJlc3VsdHM9ImhpZGUiLGluY2x1ZGU9RkFMU0V9DQoNCiMgdHJlZV90dW5lIDwtIA0KIyAgIHdvcmtmbG93X2FkdWx0X3RyZWUgJT4lIA0KIyAgIHR1bmVfZ3JpZCgNCiMgICAgIHJlc2FtcGxlcyA9IGFkdWx0X3Zmb2xkLA0KIyAgICAgZ3JpZCA9IHRyZWVfZ3JpZCwNCiMgICAgIGNvbnRyb2wgPSBjb250cm9sX2dyaWQoc2F2ZV9wcmVkID0gVFJVRSwgdmVyYm9zZSA9IEZBTFNFLCBhbGxvd19wYXIgPSBGKSwNCiMgICAgIG1ldHJpY3MgPSBtZXRyaWNfc2V0KHJvY19hdWMpDQojICAgKQ0KDQpgYGANCg0KDQpgYGB7cn0NCiMgYXV0b3Bsb3QodHJlZV90dW5lKQ0KIyBzaG93X2Jlc3QodHJlZV90dW5lLCAicm9jX2F1YyIpDQojIA0KIyB0cmVlX2Jlc3RfaGlwZXJwYXJhbXMgPC0gc2VsZWN0X2Jlc3QodHJlZV90dW5lKSAjMS4wNjk0MTVlLTA5CTgJMTkJTW9kZWwwNCAocm9jX2F1YyA9IDAuODk5MzYwOCkNCiMgdHJlZV9iZXN0X2hpcGVycGFyYW1zDQoNCg0KYGBgDQoNCkZpbmFsaXphbmRvIFdGOg0KDQpgYGB7cn0NCndvcmtmbG93X3RyZWVfZmluYWwgPC0gZmluYWxpemVfd29ya2Zsb3coDQogIHdvcmtmbG93X2FkdWx0X3RyZWUsDQogICN0cmVlX2Jlc3RfaGlwZXJwYXJhbXMNCiAgcGFyYW1ldGVycyh3b3JrZmxvd19hZHVsdF90cmVlKQ0KKQ0KDQp3b3JrZmxvd190cmVlX2ZpbmFsDQpgYGANCg0KDQpWZXJpZmljYW5kbyBpbXBvcnTDom5jaWEgZG9zIGF0cmlidXRvczoNCg0KYGBge3J9DQp3b3JrZmxvd190cmVlX2ZpbmFsICU+JQ0KICBmaXQoYWR1bHRfdHJhaW4pICU+JQ0KICBwdWxsX3dvcmtmbG93X2ZpdCgpICU+JQ0KICB2aXA6OnZpcChnZW9tID0gImNvbCIpDQpgYGANCg0KDQpNb2RlbG8gZmluYWw6DQoNCmBgYHtyfQ0KDQp0cmVlX2ZpbmFsIDwtIGxhc3RfZml0KHdvcmtmbG93X3RyZWVfZmluYWwsIGFkdWx0X3NwbGl0KQ0KY29sbGVjdF9tZXRyaWNzKHRyZWVfZmluYWwpDQoNCmBgYA0KDQoNCg0KIyMjIFJhbmRvbSBGb3Jlc3QNCg0KRXNwZWNpZmljYW5kbyBtb2RlbG86DQoNCmBgYHtyfQ0KYWR1bHRfcmYgPC0gDQogIHJhbmRfZm9yZXN0KA0KICAgIG1pbl9uID0gMjEsDQogICAgbXRyeSA9IDIzLA0KICAgIHRyZWVzID0gMTcxNSkgJT4lDQogIHNldF9tb2RlKCJjbGFzc2lmaWNhdGlvbiIpICU+JQ0KICBzZXRfZW5naW5lKCJyYW5kb21Gb3Jlc3QiKQ0KIyAyMyAgMTcxNSAgICAyMQ0KYWR1bHRfcmYNCmBgYA0KDQpXb3JrZmxvdyBwYXJhIHJhbmRvbSBmb3Jlc3Q6DQoNCmBgYHtyfQ0KDQp3b3JrZmxvd19hZHVsdF9yZiA8LSANCiAgYWR1bHRfd2YgJT4lIA0KICBhZGRfbW9kZWwoYWR1bHRfcmYpDQoNCg0KYGBgDQoNCg0KR3JpZDoNCg0KYGBge3J9DQojIHNldC5zZWVkKDMyKQ0KIyANCiMgcmZfZ3JpZCA8LSBwYXJhbWV0ZXJzKGFkdWx0X3JmKSAlPiUgDQojICAgZmluYWxpemUoYmFrZShwcmVwKGFkdWx0X3JlY2lwZSksIGFkdWx0X3RyYWluKSkgJT4lIA0KIyAgIGdyaWRfbWF4X2VudHJvcHkoc2l6ZSA9IDEwKQ0KIyANCiMgcmZfZ3JpZA0KYGBgDQoNCkVmZXR1YW5kbyB0dW5hZ2VtIGRlIGhpcGVycGFyw6JtZXRyb3M6DQoNCmBgYHtyfQ0KDQojIGxpYnJhcnkoZG9QYXJhbGxlbCkNCiMgbGlicmFyeSgiZG9GdXR1cmUiKQ0KIyANCiMgYWxsX2NvcmVzIDwtIHBhcmFsbGVsOjpkZXRlY3RDb3Jlcyhsb2dpY2FsID0gRkFMU0UpIC0gMQ0KIyByZWdpc3RlckRvRnV0dXJlKCkNCiMgY2wgPC0gbWFrZUNsdXN0ZXIoYWxsX2NvcmVzKQ0KIyBwbGFuKGZ1dHVyZTo6Y2x1c3Rlciwgd29ya2VycyA9IGNsKQ0KIyBnZXREb1BhcldvcmtlcnMoKQ0KIyANCiMgc2V0LnNlZWQoMTIzKQ0KIyByZl90dW5lPC0gDQojICAgd29ya2Zsb3dfYWR1bHRfcmYgJT4lIA0KIyAgIHR1bmVfZ3JpZCgNCiMgICAgIHJlc2FtcGxlcyA9IGFkdWx0X3Zmb2xkLA0KIyAgICAgZ3JpZCA9IHJmX2dyaWQsDQojICAgICBjb250cm9sID0gY29udHJvbF9ncmlkKHNhdmVfcHJlZCA9IFRSVUUsIHZlcmJvc2UgPSBGQUxTRSwgYWxsb3dfcGFyID0gVCksDQojICAgICBtZXRyaWNzID0gbWV0cmljX3NldChyb2NfYXVjKQ0KIyAgICkNCg0KYGBgDQoNCmBgYHtyfQ0KIyBhdXRvcGxvdChyZl90dW5lKQ0KIyBzaG93X2Jlc3QocmZfdHVuZSwicm9jX2F1YyIpDQojIA0KIyByZl9iZXN0X2hpcGVycGFyYW1zIDwtIHNlbGVjdF9iZXN0KHJmX3R1bmUpIA0KIyByZl9iZXN0X2hpcGVycGFyYW1zIA0KDQojIyAgICBtdHJ5IHRyZWVzIG1pbl9uIC5jb25maWcNCiMjICAgPGludD4gPGludD4gPGludD4gPGNocj4gIA0KIyMgMSAgICAyMyAgMTcxNSAgICAyMSBNb2RlbDAxDQpgYGANCg0KRmluYWxpemFuZG8gV0Y6DQoNCmBgYHtyfQ0Kd29ya2Zsb3dfcmZfZmluYWwgPC0gZmluYWxpemVfd29ya2Zsb3coDQogIHdvcmtmbG93X2FkdWx0X3JmLA0KICAjcmZfYmVzdF9oaXBlcnBhcmFtcw0KICBwYXJhbWV0ZXJzKHdvcmtmbG93X2FkdWx0X3JmKQ0KKQ0KDQp3b3JrZmxvd19yZl9maW5hbA0KYGBgDQoNClZlcmlmaWNhbmRvIGltcG9ydMOibmNpYSBkb3MgYXRyaWJ1dG9zOg0KDQpgYGB7cn0NCndvcmtmbG93X3JmX2ZpbmFsICU+JQ0KICBmaXQoYWR1bHRfdHJhaW4pICU+JQ0KICBwdWxsX3dvcmtmbG93X2ZpdCgpICU+JQ0KICB2aXA6OnZpcChnZW9tID0gImNvbCIpDQoNCmBgYA0KDQoNCk1vZGVsbyBmaW5hbDoNCg0KYGBge3J9DQoNCnJmX2ZpbmFsIDwtIGxhc3RfZml0KHdvcmtmbG93X3JmX2ZpbmFsLCBhZHVsdF9zcGxpdCkNCmNvbGxlY3RfbWV0cmljcyhyZl9maW5hbCkgI3JvY19hdWMgPSAwLjkwOTQyNDINCg0KYGBgDQoNCiMjIyBYZ2Jvb3N0DQoNCkNvbW8gbyBYZ2Jvb3N0IHBvc3N1aSBtdWl0b3MgcGFyw6JtZXRyb3MsIG9wdGVpIHBvcmEgbsOjbyB0dW5uYXIgb3MgcGFyw6JtZXRyb3MgYGxvc3NfcmVkdWN0aW9uYCBlIGBzYW1wbGVzX3NpemVgIG5lc3NlIHByaW1laXJvIG1vbWVudG8uIFNlbmRvIGFzc2ltIG9zIHZhbG9yZXMgZGVmYXVsdCBkYSBlbmdpbmVlIGB4Z2Jvb3N0YCBzw6NvIGF0cmlidcOtZG9zIMOgIGVzc2VzIHBhcsOibWV0cm9zLCBsb3NzX3JlZHVjdGlvbiA9IDAgZSBzYW1wbGVfc2l6ZSA9IDEuDQoNCmBgYHtyfQ0KYWR1bHRfeGdiIDwtIA0KICBib29zdF90cmVlKA0KICAgbXRyeSA9IDM0LCANCiAgdHJlZXMgPSAxMzA5LCANCiAgbWluX24gPSA1LCANCiAgdHJlZV9kZXB0aCA9IDEwLA0KICAjIGxvc3NfcmVkdWN0aW9uID0gdHVuZSgpLCANCiAgbGVhcm5fcmF0ZSA9IDAuMDEwNiwgDQogICMgc2FtcGxlX3NpemUgPSB0dW5lKCkNCiAgKSAlPiUNCiAgc2V0X21vZGUoImNsYXNzaWZpY2F0aW9uIikgJT4lDQogIHNldF9lbmdpbmUoInhnYm9vc3QiKQ0KDQojIyAgICBtdHJ5IHRyZWVzIG1pbl9uIHRyZWVfZGVwdGggbGVhcm5fcmF0ZSAuY29uZmlnDQojIyAgIDxpbnQ+IDxpbnQ+IDxpbnQ+ICAgICAgPGludD4gICAgICA8ZGJsPiA8Y2hyPiAgDQojIyAxICAgIDM0ICAxMzA5ICAgICA1ICAgICAgICAgMTAgICAgIDAuMDEwNiBNb2RlbDEzDQoNCmFkdWx0X3hnYg0KYGBgDQoNCldvcmtmbG93IHBhcmEgWGdib29zdDoNCg0KYGBge3J9DQoNCndvcmtmbG93X2FkdWx0X3hnYiA8LSANCiAgYWR1bHRfd2YgJT4lIA0KICBhZGRfbW9kZWwoYWR1bHRfeGdiKQ0KDQp3b3JrZmxvd19hZHVsdF94Z2INCmBgYA0KDQpHcmlkOg0KDQpgYGB7cn0NCiMgc2V0LnNlZWQoMzIpDQojIA0KIyB4Z2JfZ3JpZCA8LSBwYXJhbWV0ZXJzKGFkdWx0X3hnYikgJT4lDQojICAgICBmaW5hbGl6ZShiYWtlKHByZXAoYWR1bHRfcmVjaXBlKSxhZHVsdF90cmFpbikpICU+JQ0KIyAgICAgZ3JpZF9tYXhfZW50cm9weShzaXplID0gMjApDQojIA0KIyB4Z2JfZ3JpZA0KDQoNCmBgYA0KDQo8YnI+RWZldHVhbmRvIHR1bmFnZW0gZGUgaGlwZXJwYXLDom1ldHJvczo8L2JyPg0KDQpgYGB7cn0NCiMgbGlicmFyeShkb0Z1dHVyZSkNCiMgYWxsX2NvcmVzIDwtIHBhcmFsbGVsOjpkZXRlY3RDb3Jlcyhsb2dpY2FsID0gRkFMU0UpIC0gMQ0KIyANCiMgcmVnaXN0ZXJEb0Z1dHVyZSgpDQojIGNsIDwtIG1ha2VDbHVzdGVyKGFsbF9jb3JlcykNCiMgcGxhbihmdXR1cmU6OmNsdXN0ZXIsIHdvcmtlcnMgPSBjbCkNCiMgZ2V0RG9QYXJXb3JrZXJzKCkNCiMgDQojICMgZ3JpZCBzZWFyY2gNCiMgaW5pIDwtIFN5cy50aW1lKCkNCiMgeGdiX3R1bmUgPC0NCiMgICB3b3JrZmxvd19hZHVsdF94Z2IgJT4lDQojICAgICB0dW5lX2dyaWQoDQojICAgICAgICAgcmVzYW1wbGVzID0gYWR1bHRfdmZvbGQsDQojICAgICAgICAgZ3JpZCA9IHhnYl9ncmlkLA0KIyAgICAgICAgIGNvbnRyb2wgPSBjb250cm9sX2dyaWQodmVyYm9zZSA9IEZBTFNFKSwNCiMgICAgICAgICBtZXRyaWNzID0gbWV0cmljX3NldChyb2NfYXVjKQ0KIyAgICAgKQ0KIyBTeXMudGltZSgpLSBpbmkgI1RpbWUgZGlmZmVyZW5jZSBvZiAzOS45ODQ0IG1pbnMocGFyYWxsZWwpDQojIA0KIyBmb3JlYWNoOjpyZWdpc3RlckRvU0VRKCkNCmBgYA0KDQoNCmBgYHtyfQ0KIyBhdXRvcGxvdCh4Z2JfdHVuZSkNCiMgc2hvd19iZXN0KHhnYl90dW5lLCJyb2NfYXVjIikNCiMgDQojIHhnYl9iZXN0X2hpcGVycGFyYW1zIDwtIHNlbGVjdF9iZXN0KHhnYl90dW5lKQ0KIyB4Z2JfYmVzdF9oaXBlcnBhcmFtcyANCg0KIyMgICAgbXRyeSB0cmVlcyBtaW5fbiB0cmVlX2RlcHRoIGxlYXJuX3JhdGUgLmNvbmZpZw0KIyMgICA8aW50PiA8aW50PiA8aW50PiAgICAgIDxpbnQ+ICAgICAgPGRibD4gPGNocj4gIA0KIyMgMSAgICAzNCAgMTMwOSAgICAgNSAgICAgICAgIDEwICAgICAwLjAxMDYgTW9kZWwxMw0KDQpgYGANCkZpbmFsaXphbmRvIFdGOg0KDQpgYGB7cn0NCndvcmtmbG93X3hnYl9maW5hbCA8LSBmaW5hbGl6ZV93b3JrZmxvdygNCiAgd29ya2Zsb3dfYWR1bHRfeGdiLA0KICAjeGdiX2Jlc3RfaGlwZXJwYXJhbXMNCiAgcGFyYW1ldGVycyh3b3JrZmxvd19hZHVsdF94Z2IpDQopDQoNCndvcmtmbG93X3hnYl9maW5hbA0KYGBgDQoNCg0KVmVyaWZpY2FuZG8gaW1wb3J0w6JuY2lhIGRvcyBhdHJpYnV0b3M6DQoNCmBgYHtyLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQ0Kd29ya2Zsb3dfeGdiX2ZpbmFsICU+JQ0KICBmaXQoYWR1bHRfdHJhaW4pICU+JQ0KICBwdWxsX3dvcmtmbG93X2ZpdCgpICU+JQ0KICB2aXA6OnZpcChnZW9tID0gImNvbCIpDQpgYGANCg0KTW9kZWxvIGZpbmFsOg0KDQpgYGB7cn0NCg0KeGdiX2ZpbmFsIDwtIGxhc3RfZml0KHdvcmtmbG93X3hnYl9maW5hbCwgYWR1bHRfc3BsaXQpDQpjb2xsZWN0X21ldHJpY3MoeGdiX2ZpbmFsKSAjMC45MjQ2MzEwDQoNCmBgYA0KDQojIyMgWGdib29zdDINCg0KQWdvcmEgdmFtb3MgaW5zZXJpciBvcyB2YWxvcmVzIGlkZW50aWZpY2Fkb3MgbmEgdHVuYWdlbSBwYXJhIG9zIHBhcsOibWV0cm9zIGUgZWZldHVhciBvIHR1bmluZyBwYXJhIG9zIHBhcsOibWV0cm9zIHF1ZSBgc2FtcGxlX3NpemVgIGUgYGxvc3NfcmVkdWN0aW9uYDoNCg0KYGBge3J9DQphZHVsdF94Z2IyIDwtIA0KICBib29zdF90cmVlKA0KICAgbXRyeSA9IDM0LCANCiAgdHJlZXMgPSAxMzA5LCANCiAgbWluX24gPSA1LCANCiAgdHJlZV9kZXB0aCA9IDEwLA0KICBsb3NzX3JlZHVjdGlvbiA9IDAuMDAwMTI3LCN0dW5lKCksDQogIGxlYXJuX3JhdGUgPSAwLjAxMDY0NDUsIA0KICBzYW1wbGVfc2l6ZSA9IDAuOTg5LCN0dW5lKCkNCiAgKSAlPiUNCiAgc2V0X21vZGUoImNsYXNzaWZpY2F0aW9uIikgJT4lDQogIHNldF9lbmdpbmUoInhnYm9vc3QiKQ0KDQojIyAgIGxvc3NfcmVkdWN0aW9uIHNhbXBsZV9zaXplIC5jb25maWcNCiMjICAgICAgICAgICAgPGRibD4gICAgICAgPGRibD4gPGNocj4gIA0KIyMgMSAgICAgICAwLjAwMDEyNyAgICAgICAwLjk4OSBNb2RlbDA5DQoNCmFkdWx0X3hnYjINCg0KYGBgIA0KDQpXb3JrZmxvdyBwYXJhIFhnYm9vc3Q6DQoNCmBgYHtyfQ0KDQp3b3JrZmxvd19hZHVsdF94Z2IyIDwtIA0KICBhZHVsdF93ZiAlPiUgDQogIGFkZF9tb2RlbChhZHVsdF94Z2IyKQ0KDQpgYGANCg0KR3JpZDoNCg0KYGBge3J9DQojIHNldC5zZWVkKDMyKQ0KIyANCiMgeGdiX2dyaWQyIDwtIHBhcmFtZXRlcnMoYWR1bHRfeGdiMikgJT4lDQojICAgICBncmlkX21heF9lbnRyb3B5KHNpemUgPSAyMCkNCiMgDQojIHhnYl9ncmlkMg0KDQoNCmBgYA0KDQpFZmV0dWFuZG8gdHVuYWdlbSBkZSBoaXBlcnBhcsOibWV0cm9zOg0KDQpgYGB7cn0NCg0KIyBhbGxfY29yZXMgPC0gcGFyYWxsZWw6OmRldGVjdENvcmVzKGxvZ2ljYWwgPSBGQUxTRSkgLSAxDQojIA0KIyByZWdpc3RlckRvRnV0dXJlKCkNCiMgY2wgPC0gbWFrZUNsdXN0ZXIoYWxsX2NvcmVzKQ0KIyBwbGFuKGZ1dHVyZTo6Y2x1c3Rlciwgd29ya2VycyA9IGNsKQ0KIyBnZXREb1BhcldvcmtlcnMoKQ0KIyANCiMgIyBncmlkIHNlYXJjaA0KIyBpbmkgPC0gU3lzLnRpbWUoKQ0KIyB4Z2JfdHVuZTIgPC0NCiMgICB3b3JrZmxvd19hZHVsdF94Z2IyICU+JQ0KIyAgICAgdHVuZV9ncmlkKA0KIyAgICAgICAgIHJlc2FtcGxlcyA9IGFkdWx0X3Zmb2xkLA0KIyAgICAgICAgIGdyaWQgPSB4Z2JfZ3JpZDIsDQojICAgICAgICAgY29udHJvbCA9IGNvbnRyb2xfZ3JpZCh2ZXJib3NlID0gRkFMU0UpLA0KIyAgICAgICAgIG1ldHJpY3MgPSBtZXRyaWNfc2V0KHJvY19hdWMpDQojICAgICApDQojIFN5cy50aW1lKCktIGluaQ0KIyANCiMgZm9yZWFjaDo6cmVnaXN0ZXJEb1NFUSgpDQpgYGANCg0KDQpgYGB7cn0NCiMgYXV0b3Bsb3QoeGdiX3R1bmUyKQ0KIyBzaG93X2Jlc3QoeGdiX3R1bmUyLCJyb2NfYXVjIikNCiMgDQojIHhnYjJfYmVzdF9oaXBlcnBhcmFtcyA8LSBzZWxlY3RfYmVzdCh4Z2JfdHVuZTIpDQojIHhnYjJfYmVzdF9oaXBlcnBhcmFtcyANCg0KYGBgDQpGaW5hbGl6YW5kbyBXRjoNCg0KYGBge3J9DQp3b3JrZmxvd194Z2JfZmluYWwyIDwtIGZpbmFsaXplX3dvcmtmbG93KA0KICB3b3JrZmxvd19hZHVsdF94Z2IyLA0KICAjIHhnYjJfYmVzdF9oaXBlcnBhcmFtcw0KICBwYXJhbWV0ZXJzKHdvcmtmbG93X2FkdWx0X3hnYjIpDQopDQoNCndvcmtmbG93X3hnYl9maW5hbDINCmBgYA0KDQpWZXJpZmljYW5kbyBpbXBvcnTDom5jaWEgZG9zIGF0cmlidXRvczoNCg0KYGBge3J9DQoNCndvcmtmbG93X3hnYl9maW5hbDIgJT4lDQogIGZpdChhZHVsdF90cmFpbikgJT4lDQogIHB1bGxfd29ya2Zsb3dfZml0KCkgJT4lDQogIHZpcDo6dmlwKGdlb20gPSAiY29sIikNCg0KYGBgDQoNCk1vZGVsbyBmaW5hbDoNCg0KYGBge3J9DQoNCnhnYjJfZmluYWwgPC0gbGFzdF9maXQod29ya2Zsb3dfeGdiX2ZpbmFsMiwgYWR1bHRfc3BsaXQpDQpjb2xsZWN0X21ldHJpY3MoeGdiMl9maW5hbCkgIzAuOTI0MzI4NQ0KDQpgYGANCg0KIyBDb21wYXJhbmRvIG9zIG1vZGVsb3MNCg0KYGBge3J9DQpiaW5kX3Jvd3MoDQogdHJlZV9maW5hbCAlPiUNCiAgY29sbGVjdF9wcmVkaWN0aW9ucygpICU+JSANCiAgbXV0YXRlKGlkID0gIkRlY2lzaW9uIHRyZWUiKQ0KICAsDQogcmZfZmluYWwgJT4lDQogIGNvbGxlY3RfcHJlZGljdGlvbnMoKSAlPiUNCiAgbXV0YXRlKGlkID0gIlJhbmRvbSBGb3Jlc3QiKQ0KICwNCiAgeGdiX2ZpbmFsICU+JQ0KICBjb2xsZWN0X3ByZWRpY3Rpb25zKCkgJT4lDQogIG11dGF0ZShpZCA9ICJ4Z2Jvb3N0IikNCiAgLA0KICB4Z2IyX2ZpbmFsICU+JQ0KICBjb2xsZWN0X3ByZWRpY3Rpb25zKCkgJT4lDQogIG11dGF0ZShpZCA9ICJ4Z2Jvb3N0MiIpDQopICU+JSANCiAgZ3JvdXBfYnkoaWQpICU+JSANCiAgbmVzdCgpICU+JSANCiAgdW5ncm91cCgpICU+JSANCiAgbXV0YXRlKHJvYyA9IG1hcChkYXRhLCB+cm9jX2N1cnZlKC54LCB0cnV0aCA9IHJlc3Bvc3RhLCBgLnByZWRfPD01MEtgKSksDQogICAgICAgICBhdWMgPSBtYXBfZGJsKGRhdGEsIH5yb2NfYXVjKC54LCB0cnV0aCA9IHJlc3Bvc3RhLCBgLnByZWRfPD01MEtgKSAlPiUgDQogICAgICAgICAgICAgICAgICAgICAgICAgcHVsbCguZXN0aW1hdGUpICU+JSByb3VuZCgzKSksDQogICAgICAgICBpZCA9IHBhc3RlMChpZCwgIiBhdWM6ICIsIGF1YykpICU+JSANCiAgc2VsZWN0KC1kYXRhKSAlPiUgDQogIHVubmVzdChjb2xzID0gYyhyb2MpKSAlPiUgDQogIGdncGxvdCgpICsNCiAgYWVzKHggPSAxIC0gc3BlY2lmaWNpdHksIHkgPSBzZW5zaXRpdml0eSwgY29sb3IgPSBpZCkgKw0KICBnZW9tX3BhdGgoKSArDQogIGdlb21fYWJsaW5lKGx0eSA9IDMpICsNCiAgZ2d0aXRsZSgiQ3VydmEgUm9jIikNCg0KDQpgYGANCg0KUG9kZW1vcyB2ZXIgYSBwYXJ0aXIgZGEgY3VydmEgcm9jIHF1ZSBvIHhnYm9vc3Qgb2J0ZXZlIG1lbGhvciBwZXJmb21hbmNlIHF1ZSBvIHJhbmRvbSBmb3Jlc3QgZSBhIMOhcnZvcmUgZGUgZGVjaXPDo28uIA0KSW50ZXJlc3NhbnRlIHF1ZSBvIHhnYm9vc3Qgc2VtIHR1bmFyIG9zIGhpcGVycGFyw6JtZXRyb3MgYGxvc3NfcmVkdWN0aW9uYCBlIGBzYW1wbGVfc2l6ZWAgc2Ugc2FpdSBkaXNjcmV0YW1lbnRlIG1lbGhvciBxdWUgbyB4Z2Jvb3N0MiBvbmRlIGVmZXR1YW1vcyBvIHR1bmluZyBkZXNzZXMgZG9pcyBoaXBlcnBhcsOibWV0cm9zLiBTZW5kbyBhc3NpbSBub3NzbyBtb2RlbG8gZmluYWwgc2Vyw6EgbyAqKnhnYl9maW5hbCoqLg0KDQojIFNjb3JhZ2VtIHBhcmEgc3VibWV0ZXIgcmVzdWx0YWRvDQoNClZhbW9zIGVudMOjbyBmaW5hbGl6YXIgbm9zc28gbW9kZWxvIGNhbXBlw6NvIGUgc2NvcmFyIGEgYmFzZSBkZSB2YWxpZGHDp8OjbyBwYXJhIGVmZXR1YXIgYSBzdWJtaXNzw6NvOg0KDQpgYGB7cn0NCg0KYWR1bHRfdmFsIDwtIHJlYWRyOjpyZWFkX3JkcygiYWR1bHRfdmFsLnJkcyIpDQoNCnhnYm9vc3RfbW9kZWxvX2ZpbmFsIDwtIGFkdWx0X3hnYiAjJT4lIA0KICAgICMgZmluYWxpemVfbW9kZWwoeGdiX2Jlc3RfaGlwZXJwYXJhbXMpDQoNCmFkdWx0X2ZpdCA8LSANCiAgZml0KHhnYm9vc3RfbW9kZWxvX2ZpbmFsLA0KICAgIGZvcm11bGEgPSByZXNwb3N0YSB+IC4sICANCiAgICBkYXRhID0gYmFrZShwcmVwKGFkdWx0X3JlY2lwZSksIG5ld19kYXRhID0gYWR1bHQpKQ0KDQphZHVsdF92YWwkbW9yZV90aGFuXzUwayA8LSANCiAgcHJlZGljdChhZHVsdF9maXQsIA0KICAgICAgICAgIGJha2UocHJlcChhZHVsdF9yZWNpcGUpLCBuZXdfZGF0YSA9IGFkdWx0X3ZhbCksDQogICAgICAgICAgdHlwZSA9ICJwcm9iIikkYC5wcmVkXz41MEtgDQpgYGANCg0KTWF0cml6IGRlIGNvbmZ1c8OjbzoNCg0KYGBge3J9DQoNCmFkdWx0X3ZhbCAlPiUgDQogIHRyYW5zbXV0ZShyZXNwb3N0YSA9IGZhY3RvcihyZXNwb3N0YSwgbGV2ZWxzID0gYygiPjUwSyIsICI8PTUwSyIpKSwgDQogICAgICAgICAgICBtb3JlX3RoYW5fNTBrID0gaWZlbHNlKG1vcmVfdGhhbl81MGsgPiAwLjUsICI+NTBLIiwgIjw9NTBLIikgJT4lIA0KICAgICAgICAgICAgICBmYWN0b3IobGV2ZWxzID0gYygiPjUwSyIsICI8PTUwSyIpKSkgJT4lIA0KICB0YWJsZSgpICU+JSANCiAgY2FyZXQ6OmNvbmZ1c2lvbk1hdHJpeCgpDQoNCg0KYGBgDQoNCg0KU2VsZWNpb25hbmRvIGNhbXBvcyBubyBmb3JtYXRvIGRhIHN1Ym1pc3PDo286DQoNCmBgYHtyfQ0Kc3VibWlzc2FvIDwtIGFkdWx0X3ZhbCAlPiUgc2VsZWN0KGlkLCBtb3JlX3RoYW5fNTBrKQ0Kd3JpdGVfY3N2KHN1Ym1pc3NhbywgInN1Ym1pc3Nhby5jc3YiKQ0KYGBgDQoNCg0KDQoNCg0K